BY 



CLASi 



SUBCLASS 



• 




1/11 



Digital Video Source 



100 



110A 
160 




Digital Video Signal 



J- 



110 



Video 
Signal 



Audio 
Signal 



Other 
Info. 



hob 



hoc 



Digital 
Video 

Interpretation 
System 



Video Segmenter 



I 



120 



(Video \ f 
Segments p 
or Shots J 



130 



3 



Video Segment 
Analyser 



140 



Sequence of labelled 
RAGs for 
video sequence. 




150 



FIG. 1 



CFP1415AU & US IPR16 472779 



[l:\ELEC\CISRA\IPR\IPR16] 472779.fm 



*\ Y 



A CI 



^{SUBCLASS 



2/11 



Select an initial frame in 
video segment for analysis. 



200 



Frame Event Analyser: 
Use temporal ROIs to 
generate a labelled RAG 



202 



210 



Get next frame 
in video segment 
to analysed. 




z 



208 



End video 

segment 

processing 



FIG. 2 



CFP1415AU & US IPR16 472779 



[l:\ELEC\CISRA\IPR\IPR16] 472779.fm 




CFP1415AU & US IPR16 472779 



[l:\ELEC\CISRA\IPR\IPR16] 472779.fm 



, ia'AS.'*Tsi;BCLAss| 

5 



4/11 



Current Video Frame & ROI 
for each of the information 
sources (e.g., image stream, 
audio stream, etc.) 



400 



I 



Contextual Analyser 
(using ROI) 



410 



Video Frame 
Segmenter 




460 



470 



Labelled 
RAG 



480 



< 



420 



Keywords 
/Ptirases 



3 




Adjusting unit for 
adjustment of prior 
probabilities of labels 
in application domain. 



430 



440 



Application 
Domain (e.g. 
outdoor scenes) 



FIG. 4 



CFP1415AU & US IPR16 472779 



[l:\ELEC\CISRA\IPR\IPR16] 472779.fm 



BY 



ialselsuBGLASS 



5/11 



8 

w 
in 

U 

s 

O 

in 

v0 



Outdoor 
Scene 



Water Scene — 
Road Scene 

Mountain Scene 



FIG. 5 



Beach Scene 
River Scene 

Ocean Scene 



CFP1415AU & US IPR16 472779 



[l:\ELEC\CISRA\IPR\IPR16] 472779.fm 



S jcKcpUBCLASS 



6/11 



Current Video Frame & ROI 
for each of the information 
sources (e.g., image stream, 
audio stream, etc.) 



400 



410 



420 



Contextual Analyser 
(using ROI) 



Frame 
Segmenter 



Region 
Analyser 



c 



I 



Labelled 
RAG 




450 



460 



Select most probable 
application domain 
from a domain 
database. 



530- 



540 



532 



Application 
Domain 
Database 



Application domain 
selected for use (e.g. 
outdoor scenes) 




470 



480 



FIG. 6 



CFP1415AU & US IPR16 472779 



[l:\ELEC\CISRA\IPR\IPR16] 472779.fm 



'APPROVE fC*. & pfU 



>' rT SUGCLASS 




7/11 



r 



c 



ROI 



> 



^ 603 

Other 

contextual 

information 



602 



C 



Audio ROI 



> 



604 



^ Audio ^ 
Signal 



Video ROI 



> 



605 



-► Video ^ 
Signal 



Current Frame 



601 



606^ 



FIG. 7 



Node or 
Region 


Associated Cliques 


Rl 


{Rl}, {R1.R2}, {R1.R3} 


R2 


{R2}, {R2, R4}, {R2, R8}, {R2, R4, R8} 


R3 


{R3}, {R3, R4}, {R3, R7}, {R3, R4, R7} 


R4 


{R4}, {R4, R5}, {R4, R6}, {R4, R7}, {R4, R8} 


R5 


{R5} 


R6 


{R6} 


R7 


{R7} 


R8 


{R8}, {R8,R9} 


R9 


{R9} 



FIG. 9 



CFP1415AU & US IPR16 472779 



[l:\ELEC\CISRA\IPR\IPR16] 472779.fm 



BY 




8/11 



Current information for 
Video Frame (e.g. audio ROI 
motion analysis ROI, 
infrared spectral ROI) 



400 



410 



Contextual Analyser 



Audio 

Analysing 

Unit 



710 



Motion 

Analysing 

Unit 



(Keywords j , 
/PKrases J 



Infrared 
Spectral 
Analysing 
Unit 



730 



720 



420 



430 



Adjusting unit for 
adjustment of prior 
probabilities of labels. 



440 




Application 
Domain 



FIG. 8 



CFP1415AU & US IPR16 472779 



[l:\ELEC\CISRA\IPR\IPR16] 472779.fm 



9/11 



.900 



904 



1000 



Video 




Display 







906 



Digital Video 
Source 



908 910 



■902 



Video 




I/O 




Storage 


Interface 




Interface 




Device 



918 



Processor 




I/O 




Memory 






Interface 





914 



916 



Keyboard 



920- 



922 



FIG. 10 



CFP1415AU & US IPR16 472779 



[l:\ELEC\CISRA\IPR\IPR16] 472779.fm 





SUBCLASS 


DRAFTCM/.?:! 






10/11 



1000 



1002 



1004 



Video 
Capture 
Unit 



Focal 

data& 

settings 



Processing 
Unit 



1006 



User Interface 
Unit 



1012 



Sensors 

including 

audio 

microphone 



1008 



Memory 



Communications 
Port 



1010 



T 



1 



900 



Computer 



FIG. 11 



CFP1415AU & US IPR16 472779 



[I :\ELEC\CI SRAM PRMPR1 6] 472779 .fm 



IdRAF. ! ! 

11/11 





Select an initial 
frame in video 
segment for analysis. 



250 



Video Frame Analyser: 
Use temporal ROIs to 
generate RAGref. 



252 



Region Encoder: 
encode regions in 
RAG 



254 




258 



End video 
segment 
processing 



Get next frame in 
video segment 



Store/encode 
motion model 
parameters 

— I 



Yes 



266 



I 



260 



Motion 
detector 



262 




FIG. 12 



CFP1415AU & US IPR16 472779 



[l:\ELEC\CISRA\IPR\IPR16] 472779.fm 



